Respiratory state estimating device, portable device, wearable device, medium, respiratory state estimating method and respiratory state estimator

ABSTRACT

Provided is a respiratory state estimating device including a pulse wave signal acquiring unit that acquires a pulse wave signal from a portion of a living subject, a pulse rate calculating unit that calculates a pulse rate of the living subject based on the pulse wave signal, and a respiratory state estimating unit that estimates a respiratory state of the living subject based on the pulse rate. Also, provided is a respiratory state estimating method including optically acquiring a pulse wave signal from a portion of a living subject, calculating a pulse rate of the living subject based on the pulse wave signal, estimating a respiratory state of the living subject from the pulse rate.

The present application is a continuation of U.S. patent applicationSer. No. 15/628,663 filed Jun. 21, 2017, the entirety of which isexplicitly incorporated herein by reference.

The contents of the following Japanese patent application(s) areincorporated herein by reference:

-   -   NO. 2014-261089 filed in JP on Dec. 24, 2014, and    -   NO. PCT/JP2015/085889 filed on Dec. 22, 2015

BACKGROUND 1. Technical Field

The present invention relates to a respiratory state estimating device,a portable device, a wearable device, a medium, a respiratory stateestimating method and a respiratory state estimator.

2. Related Art

A conventional respiratory state estimating device includes a deviceusing an electrocardiogram and a device using pulse waves. When anelectrocardiogram is used, heartbeat intervals are measured from theelectrocardiogram, and a respiratory state is estimated from a heartbeatvariation which is a fluctuation of the heartbeat intervals. Also, whenpulse waves are used, a baseline variation of the pulse waves throughthe envelope analysis is extracted, and a respiratory pattern ismeasured by measuring the pattern of the baseline variation (see, forexample, Patent Document 1).

Patent Document 1: Japanese Patent No. 4581480 publication

However, when an electrocardiogram is used, there is a problem that alot of equipment is required and the measurement imposes a burden on asubject. Also, when pulse waves are used, a respiratory state cannot beestimated from the baseline variation in a short time.

SUMMARY

According to a first aspect of the present invention, provided is arespiratory state estimating device including a pulse wave signalacquiring unit that acquires a pulse wave signal from a portion of aliving subject, a pulse rate calculating unit that calculates a pulserate of the living subject based on the pulse wave signal, and arespiratory state estimating unit that estimates a respiratory state ofthe living subject based on the pulse rate.

According to a second aspect of the present invention, provided is aportable device including the respiratory state estimating deviceaccording to the first aspect and a display that displays informationindicating the respiratory state.

According to a third aspect of the present invention, provided is awearable device including the respiratory state estimating deviceaccording to the first aspect and a display that displays informationindicating the respiratory state.

According to a fourth aspect of the present invention, provided iscomputer readable medium having a program recorded thereon that causes acomputer to function as the respiratory state estimating deviceaccording to the first aspect.

According to a fifth aspect of the present invention, provided is arespiratory state estimating method including optically acquiring apulse wave signal from a portion of a living subject, calculating apulse rate of the living subject based on the pulse wave signal, andestimating a respiratory state of the living subject from the pulserate.

According to a sixth aspect of the present invention, provided is arespiratory state estimator including a video acquiring unit thatacquires a video of skin of a living subject, and a respiratory stateestimating unit that estimates a respiratory state of the living subjectbased on the video.

According to an seventh aspect of the present invention, provided is arespiratory state estimating device including a pulse wave signalacquiring unit that optically acquires a pulse wave signal from aportion of a living subject, a pulse wave lag time calculating unit thatcalculates a pulse wave lag time of the living subject based on thepulse wave signal, and a respiratory state estimating unit thatestimates a respiratory state of the living subject based on the pulsewave lag time.

The summary clause does not necessarily describe all necessary featuresof the embodiment s of the present invention. The present invention mayalso be a sub-combination of the features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an overview of a configuration of a respiratory stateestimating device 100.

FIG. 2 shows an overview of a configuration of a pulse wave signalacquiring unit 10.

FIG. 3 shows one example of an algorithm of a signal processing in thepulse wave signal acquiring unit 10.

FIG. 4 shows one example of a method of segmenting out a window signal.

FIG. 5 shows one example of a configuration of a pulse rate calculatingunit 20.

FIG. 6 shows one example of an algorithm of a signal processing in thepulse rate calculating unit 20.

FIG. 7 shows one example of the Hanning window function.

FIG. 8 shows one example of the Kaiser-Bessel-Derived window function.

FIG. 9 shows one example of a respiratory state estimating methodaccording to Comparative Example.

FIG. 10 shows correlation between a respiratory period and a pulse rate.

FIG. 11 shows one example of a respiratory state estimating method.

FIG. 12 shows one example of a respiratory state estimating method.

FIG. 13 shows relationship between a pulse rate variation and a pulserate variation speed.

FIG. 14 shows one example of a configuration of the respiratory stateestimating device 100.

FIG. 15 shows one example of a configuration of a pulse wave lag timecalculating unit 40.

FIG. 16 shows one example of an algorithm of a signal processing in thepulse wave lag time calculating unit 40.

FIG. 17 shows one example of a calculation method of a pulse wave lagtime.

FIG. 18 shows one example of continuous pulse wave lag time variation.

FIG. 19 shows one example of a respiratory state estimating method.

FIG. 20 shows the respiratory state estimating device 100 according toEmbodiment 1.

FIG. 21 shows the respiratory state estimating device 100 according toEmbodiment 2.

FIG. 22 shows one example of fixed resampling using a lightingapparatus.

FIG. 23 shows one example of a hardware configuration of a computer1900.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present invention is described through the embodimentsof the invention. However, the following embodiments do not limit theinvention according to the claims. Also, all of combinations of featuresdescribed in the embodiments are not necessarily required for a meansfor solving problems of the invention.

(Implementation 1)

FIG. 1 shows an overview of a configuration of a respiratory stateestimating device 100. The respiratory state estimating device 100includes a pulse wave signal acquiring unit 10, a pulse rate calculatingunit 20, and a respiratory state estimating unit 30. The respiratorystate estimating device 100 estimates a respiratory state based on apulse wave signal acquired from a portion of a living subject 1. Forexample, the respiratory state refers to an inspiration state, anexpiration state, an apneic state, and an instantaneous apneic state.

The pulse wave signal acquiring unit 10 acquires a pulse wave signalfrom the video of the portion of the living subject 1. The pulse wavesignal is an RGB signal or a YCbCr signal of the video including pulsewave information. The pulse wave information is information about atemporal waveform representing pulsation of the blood vessel at theportion of the living subject 1. The pulse wave information includesinformation about the timing at which a pulse wave shows a peak.

For example, the pulse wave signal acquiring unit 10 optically acquiresa pulse wave signal from a video of the portion of the living subject 1.The method of optically acquiring a pulse wave signal includes a methodusing a camera video and a method using a photoplethysmogram (PPG:Photoplethysmography). In the method using a camera video, a respiratorystate is estimated based on blood flow information included in the videoobtained by shooting the portion of the living subject 1. Alternatively,the respiratory state may be estimated based on change in thecontrasting density of the video. In the method using a PPG, therespiratory state is estimated based on change in the blood flow volumewhich can be acquired by utilizing the light absorbing property ofhemoglobin. The pulse wave signal acquiring unit 10 outputs the acquiredpulse wave signal to the pulse rate calculating unit 20.

The pulse rate calculating unit 20 calculates the pulse rate of theliving subject 1 based on the input pulse wave signal. The pulse rate ofthe living subject 1 is calculated by using a prescribed signalprocessing algorithm. The pulse rate calculating unit 20 mayperiodically calculate the pulse rates at predetermined intervals.

The pulse rate calculating unit 20 outputs the calculated pulse rate tothe respiratory state estimating unit 30.

The respiratory state estimating unit 30 estimates the respiratory stateof the living subject 1 based on the input pulse rate. For example, therespiratory state estimating unit 30 estimates the respiratory state ofthe living subject 1 based on a comparison of the calculated pulse ratewith the next calculated pulse rate. As seen above, the respiratorystate estimating unit 30 estimates the respiratory state of the livingsubject 1 by comparing the adjacent pulse rates. Therefore, therespiratory state of the living subject 1 can be estimated in real time.Alternatively, the respiratory state estimating unit 30 may estimate therespiratory state of the living subject 1 based on a comparison of thecalculated pulse rate with the calculated pulse rate after the next. Inthis case, compared with the case where the respiratory state isestimated from a baseline signal, the respiratory state estimating unit30 can estimate the respiratory state in a short time because theprocessing of extracting the baseline signal through envelope detectionhaving a large time constant is not required. Alternatively, therespiratory state estimating unit 30 may estimate the respiratory statefrom change in the pulse rate. For example, the respiratory stateestimating unit 30 may estimate that the case where the amount of changein the pulse rate is more than or equal to a predetermined positiveinspiration estimation threshold is the inspiration state, the casewhere the amount of change in the pulse rate is less than or equal to apredetermined negative expiration estimation threshold is the expirationstate, and the case where the amount of change in the pulse rate is notmore than or equal to the inspiration estimation threshold and is notless than or equal to the expiration estimation threshold is theinstantaneous apneic state. As seen above, the respiratory stateestimating unit 30 can perform an estimation of a respiratory state witha high accuracy by using the amount of change in the pulse rate. Inother words, the respiratory state estimating unit 30 can performidentification among the inspiration state, the expiration state, andthe instantaneous apneic state with a high accuracy. Furthermore, therespiratory state estimating unit 30 only uses the amount of change inthe pulse rate, and the respiratory state can be estimated in a shorttime.

FIG. 2 shows an overview of a configuration of a pulse wave signalacquiring unit 10. The pulse wave signal acquiring unit 10 includes avideo acquiring unit 11, a trace signal generating unit 12, a windowsegmenting unit 13, and a signal correcting unit 14.

The video acquiring unit 11 acquires the video of the portion of theliving subject 1. For example, the video acquiring unit 11 has a camera,and shoots the video of the portion of the living subject 1. The videoof the living subject 1 may be a sequence of still images or a movingimage. Alternatively, the video acquiring unit 11 may irradiate theliving subject 1 with light and acquire the reflected light. In thiscase, the video acquiring unit 11 has a light emitting diode and a photodiode.

The trace signal generating unit 12 detects a region of the livingsubject 1 to be measured based on the video acquired by the videoacquiring unit 11. The trace signal generating unit 12 traces a pulsewave signal in the detected region. The trace signal generating unit 12outputs the generated trace signal to the window segmenting unit 13.

The window segmenting unit 13 segments out a trace signal with apredetermined window size. The trace signal that has been segmented outis referred to as a window signal herein. Also, the window size refersto the time width of the window signal. The window segmenting unit 13segments out the window signals at predetermined intervals. The windowsegmenting unit 13 outputs the window signal that has been segmented outto the signal correcting unit 14.

The signal correcting unit 14 corrects the window signal. The correctionof the window signal includes interpolation of a signal and eliminationof unnecessary frequencies. For example, the signal correcting unit 14generates a processing window pulse wave the sampling rate of which isfixed at a predetermined sampling rate based on a reference signalindicating time. The processing window pulse wave is generated for eachwindow signal that has been segmented out. The signal correcting unit 14outputs the corrected signal to the pulse rate calculating unit 20 asthe processing window pulse wave.

FIG. 3 shows one example of an algorithm of a signal processing in thepulse wave signal acquiring unit 10. Through the algorithm according tothe present example, the pulse wave signal acquiring unit 10 extractsthe processing window pulse wave from the camera video. The stableextraction of the processing window pulse wave is the base technologyrequired for estimating the respiratory state correctly.

At step S100, the video acquiring unit 11 acquires the video of theportion of the living subject 1. The video of the portion of the livingsubject 1 is shot at a frame rate of about 30 times per second (30 fps).For example, the video acquiring unit 11 observes skin blood flow fromthe video of the portion of the living subject 1. The light absorptioncharacteristic of the G component (green component) of the RGBcomponents of light changes according to the hemoglobin concentration ofthe blood of the living subject 1. Because the pulse wave corresponds tovariation of the blood flow volume, the period of the variation of the Gcomponent of light transmitted through or reflected on the livingsubject 1 corresponds to the period of the pulse wave of the livingsubject 1. In other words, the video of the portion of the livingsubject 1 includes the variation waveform of the G component accordingto the pulse wave. Then, the trace signal generating unit 12 extracts anRGB signal from the acquired video of the person to be measured. Thevideo of the person to be measured in the present example has 640×480pixels.

At step S101, the trace signal generating unit 12 converts the extractedRGB signal into a YCbCr signal. Here, Y is a luminance signal, and Cband Cr are color-difference signals.

At step S102, the trace signal generating unit 12 detects a face regionand a region of interest ROI based on the luminance signal Y. The regionof interest ROI is identified based on the luminance signal Y. Theregion of interest ROI is a region in which blood vessels concentrate tothe extent that change in a color-difference signal including pulse waveinformation can be detected. For example, the trace signal generatingunit 12 can detect the processing window pulse wave having a high S/Nratio by detecting, as the portion of the living subject 1, a noseregion dense with capillaries.

At step S103, the trace signal generating unit 12 extracts the region ofinterest ROI based on the position information and the size informationof the region of interest ROI detected at step S102. Also, the tracesignal generating unit 12 acquires a Cb+Cr signal in the extractedregion of interest ROI. The region of interest ROI in the presentexample is a region of 50×50 pixels.

At step S104 the trace signal generating unit 12 performs the Gaussianfiltering on the region of interest ROI based on the acquired Cb+Crsignal. Signals from regions other than the region of interest ROIassociated with the movement of the living subject 1 are mixed into theperipheral portion of the region of interest ROI. The peripheral portionof the region of interest ROI can be suppressed by increasing theintensity of the central portion of the region of interest ROI throughthe Gaussian filtering. In other words, the low-reliability signals atthe periphery of the region of interest ROI are filtered through theGaussian filtering.

At step S105, a Cb+Cr trace signal obtained by plotting a value at anarbitrary clock time based on the filtered signal is created. The amountof computation is reduced by adopting the Cb+Cr trace signal, and thewaveform of the pulse wave can be extracted stably. For example, theCb+Cr trace signal is a value obtained by summing the Cb+Cr signals inrespective pixels over the entire region of interest ROI. Alternatively,the Cb+Cr trace signal is the average of the Cb+Cr signals in therespective pixels. That is, the Cb+Cr trace signal is set so that asingle value is obtained for the region of interest ROI.

At step S106, the window segmenting unit 13 segments out a window signalfrom the Cb+Cr trace signal. The window signal is segmented out in apredetermined window size and period.

At step S107, the signal correcting unit 14 corrects fluctuation of aframe rate of the camera frame into a fixed sampling rate by the splineinterpolation. A reference signal included in the video is used tocorrect the fluctuation of the frame rate. The reference signal is asignal indicating correct clock time at which the video acquiring unit11 acquires the video of the living subject 1. For example, thereference signal is a timestamp included in the video frame.

At step S108, a wavelength range other than the pulse wave component iscut through a band-pass filter BPF. The Cb+Cr trace signal can include alow frequency signal corresponding to external environment or themovement of the living subject. For this reason, the band-pass filterBPF cuts the wavelength range other than 0.75 Hz-4 Hz (pulse rates of45-240) corresponding to the pulse rate HR of a general living subject1. Thereby, noises other than the pulsebeat of the living subject 1 canbe cut.

FIG. 4 shows one example of a method of segmenting out a window signal.The window segmenting unit 13 segments out multiple window signals fromthe Cb+Cr trace signal so that each window signal overlaps atpredetermined time intervals. A first window signal is an immediatewindow signal. Window signals adjacent to the first window signal aredefined as a second to fourth window signals, respectively. A shiftbetween adjacent window signals is referred to as an overlap shift timeherein.

The overlap shift time is equal to a period for which the pulse ratecalculating unit 20 calculates a pulse rate. In other words, the pulserate calculating unit 20 calculates the pulse rate every shift amount ofthe overlap time. The overlap shift times in the present example areequal to each other. For example, when the respiratory period is 15seconds, the overlap shift time is set to 1 second. It is preferablethat the overlap shift time be shorter than the half of the respiratoryperiod.

The window size may be set to any size. The window size in the presentexample is about 200 frames. In this regard, it is preferable that thewindow size be larger than or equal to the pulse period of the livingsubject 1. The pulse period is time required for 1 beat of a pulsebeat.

FIG. 5 shows one example of a configuration of the pulse ratecalculating unit 20. The pulse rate calculating unit 20 includes awindow function multiplying unit 21, an integrating and outputting unit22, and a discrete frequency transforming unit 23.

The window function multiplying unit 21 multiplies the input processingwindow pulse wave by a predetermined window function. The windowfunction may be a function which is commonly used for signal processingsuch as the Hanning window, the Kaiser-Bessel-Derived window, theGaussian window, the Hamming window, the Tukey window, and the Blackmanwindow. The window function multiplying unit 21 outputs the processingwindow pulse wave that has been processed by being multiplied by thewindow function as a window processing performed pulse wave to theintegrating and outputting unit 22.

The integrating and outputting unit 22 generates an integrated windowsignal obtained by integrating sample data into the input windowprocessing performed pulse wave. The sample data may be integratedbefore, after, or before and after the window signal multiplied by thewindow function. For example, when the integrating and outputting unit22 performs zero extension on the window processing performed pulsewave, the sample data is zero. The resolution of the window processingperformed pulse wave is improved by performing the zero extension on thewindow processing performed pulse wave. The integrating and outputtingunit 22 outputs the generated integrated window signal to the discretefrequency transforming unit 23.

The discrete frequency transforming unit 23 performs the discretefrequency transformation on the integrated window signal output by theintegrating and outputting unit 22 to calculate a pulse wave featureamount. The pulse wave feature amount is an FFT spectrum obtained byperforming fast Fourier transform (FFT: Fast Fourier Transform) on theintegrated window signal. The discrete frequency transforming unit 23can calculate a high-resolution pulse rate based on the FFT spectrumobtained from the high-resolution integrated window signal. The discretefrequency transforming unit 23 outputs the calculated pulse rate to therespiratory state estimating unit 30.

FIG. 6 shows one example of an algorithm of a signal processing in thepulse rate calculating unit 20. The processing window pulse wave whichis input to the pulse rate calculating unit 20 is a high-reliabilitypulse wave obtained by eliminating unnecessary components by means ofthe pulse wave signal acquiring unit 10. The pulse rate calculating unit20 performs the processing at steps S201 to S203 to calculate anaccurate pulse rate using the processing window pulse wave that has beensegmented out.

At step S201, the window function multiplying unit 21 performs thewindow processing on the processing window pulse wave using the Hanningwindow function or the Kaiser-Bessel-Derived window function. Thisallows temporal weighting. Also, the window function may be selected sothat pulse intensities at both ends of the processing window pulse wavebecome equal.

At step S202, the integrating and outputting unit 22 integrates thesample data after the window processing performed pulse wave to generatean integrated window signal. For example, the sample data is data whichis equal to the pulse intensities at both ends of the processing windowpulse wave after multiplication by the window function. In this case,the sample data in the present example is zero. Also, the zero extensionis performed on the size of the integrated window signal to be a size ofpower of two. The resolution can be improved compared with that beforethe integration of the sample data by performing the zero extension.

At step S203, the discrete frequency transforming unit 23 performs theFFT on the integrated window signal and calculates an FFT spectrum. Thefrequency resolution of the FFT Af depends on the number of samples Nand the sampling rate fs and is determined by Δf=fs/N. Therefore, as thenumber of samples N increases, the resolution Δf is improved.

For example, if the frequency analysis by the FFT is performed directlywithout the zero extension on the window signal having the number ofpoints of 128, the frequency resolution is 0.23 Hz. Because itcorresponds to the pulse rate of 14 bpm, the pulse rate variationsmaller than this rate cannot be detected. On the other hand, when thenumber of samples is increased to 1024 by adding 896 zero signals to thesame window signal having the number of points of 128, the frequencyresolution becomes 0.029 Hz. It corresponds to the pulse rate of 1.7bpm. Although the number of samples after the zero extension is notlimited, it is preferably a power of two, more preferably, 256, 512,1024, 2048, or 4096.

As described above, the pulse rate calculating unit 20 calculates theintegrated window signal from the processing window pulse wave acquiredat a low sampling rate of 30 Hz. For this reason, when the pulse wavesignal is optically acquired, a high-resolution pulse rate variation canbe measured without upsampling of the sampling frequency. By using ahigh-resolution pulse rate, the estimation accuracy of the respiratorystate is improved.

FIG. 7 shows the Hanning window function. The Hanning window function isone example of the window function for the FFT. The Hanning windowfunction is a window function in which both ends of the frame are zero.Also, the Hanning window function w(n) is expressed by the followingequation (equation 1).

${{\omega(n)} = {0.5\left( {1 - {\cos\left( {2\pi\frac{n}{N}} \right)}} \right)}},{0 \leq n \leq N}$

Here, n represents a sample element, and N represents the number ofsamples.

The Hanning window function is a function having a weight at the windowcenter time (around the number of frames of 64). For this reason, thepulse rate is measured with the pulse wave at the window center time asa center. For example, when the pulse rate measurement is performedthrough the FFT with the window size set to the number of samples of 128at a frame rate of 30 Hz, in the case of the Hanning window, the pulserate measurement is performed with the pulse wave about 4 seconds beforeas a center. That is, the time difference between the time of pulse ratemeasurement and the center time can cause response time.

FIG. 8 shows one example of the Kaiser-Bessel-Derived window functionKBD (Kaiser-Bessel-Derived Window). As with the Hanning window function,the KBD window function is a window function which makes both ends ofthe frame zero.

The KBD window function d_(k) is expressed by the following equation(equation 2) using a formula in the Kaiser window w_(k) term.

$d_{k} = \left\{ \begin{matrix}\sqrt{\frac{\sum_{j = 0}^{k}\omega_{j}}{\sum_{j = 0}^{n}\omega_{j}}} & {{{if}\mspace{14mu} 0} \leq k < n} \\\sqrt{\frac{\sum_{j = 0}^{{2n} - 1 - k}\omega_{j}}{\sum_{j = 0}^{n}\omega_{j}}} & {{{if}\mspace{14mu} n} \leq k < {2n}} \\0 & {{{{if}\mspace{14mu} k} \leq 0},{{2n} \leq k}}\end{matrix} \right.$

The (equation 2) defines the window having a length of 2n. Here, d_(k)meets a next Princen-Bradley condition for modified discrete cosinetransform (MDCT: modified discrete cosine transform). That is, d_(k) isexpressed by d_(k) ²+d_(k+n) ²=1 when w_(n−k)=w_(k). Also, the KBDwindow meets the symmetry d_(k)=d_(2n−1−k) which is another MDCTcondition.

For the KBD window function, weighting around the number of frames of 40to 90 is large. On the other hand, for the Hanning window function,weighting is concentrated around the number of frames of 64. Therefore,the KBD window function puts higher weight to a pulse wave close to anewer extracted sample than the Hanning window function. For thisreason, with the KBD window, a value of the pulse wave close to a newerextracted sample is easily reflected, and the response of therespiratory state estimation can be improved.

COMPARATIVE EXAMPLE

FIG. 9 shows one example of a respiratory state estimating methodaccording to Comparative Example. The portion (a) of FIG. 9 shows changein the pulse wave over time. The portion (b) of FIG. 9 shows change inthe pulse wave over time and baseline variation. The portion (a) of FIG.9 shows a waveform obtained by enlarging a part of the pulse wave of theportion (b) of FIG. 9. In FIG. 9, the period t represents a period ofthe pulse wave, the period T represents a period of the baselinevariation. In the respiratory state estimating method according toComparative Example, a respiratory state is estimated by utilizing thefact that the period T of the baseline variation corresponds to therespiratory period.

For example, while the period t of the pulse wave is about 1 second, theperiod T of the baseline variation is about 15 seconds. Also, in therespiratory state estimating method according to Comparative Example, arespiratory state is estimated according to a baseline variationpattern. For this reason, in the respiratory state estimating methodaccording to Comparative Example, a respiratory state cannot beestimated only one time per about 15 seconds. As seen above, for themethod according to Comparative Example, a respiratory state cannot beestimated in real time and smooth estimation of a respiratory statecannot be realized.

Also, it is estimated that the baseline variation is induced due torespiration when there is no disturbance such as movement of the body ofthe subject. Moreover, the period T of the baseline variation has afrequency closer to a noise caused when an image of the living subject 1is optically acquired than to the period t of the pulse wave. Therefore,for the respiratory state estimating method according to ComparativeExample, the effect of the error caused by movement of the subject andthe error caused when acquiring a video is large, and a respiratorystate cannot be estimated correctly.

FIG. 10 shows relationship between a respiratory period and a pulserate. The subject in the present example changes his/her respiratoryperiod into three stages that are 3 bpm, 6 bpm, and 12 bpm, in additionto a normal state and a breath hold state.

The respiratory state estimating unit 30 calculates a high-resolutioncontinuous pulse rate variation from the pulse rate output by the pulserate calculating unit 20. The continuous pulse rate variation is tracedby plotting according to periodic measurement of pulse rates. Then, therespiratory state estimating unit 30 estimates a respiratory state fromthe continuous pulse rate variation.

The normal state is a state in which the living subject 1 repeatsinspiration and expiration as per usual. The inspiration and expirationof the living subject 1 is linked with the pulse rate variation.Specifically, when the living subject 1 inspires, the pulse rateincreases and when the living subject 1 expires, the pulse ratedecreases. Thus, a respiratory state can be detected in real time byobserving increase and decrease in the pulse rate.

The breath hold state is equivalent to a state in which the subject doesnot inspire or expire, or a state in which the subject hardly inspiresand expires. That is, the breath hold state corresponds to so-calledapneic state. The pulse rate variation in the breath hold state is smallcompared with that in the term of the normal state.

A respiratory period of 3 bpm refers to repeating inspiration andexpiration 3 times per 1 minute. In the present example, the inspirationtime and the expiration time are equal to each other, and each of themis 10 seconds. Similarly, a respiratory period of 6 bpm refers torepeating inspiration and expiration 6 times per 1 minute. For therespiratory period 6 bpm, the inspiration time and the expiration timeare 5 seconds, respectively. A respiratory period of 12 bpm refers torepeating inspiration and expiration 12 times per 1 minute. For therespiratory period 12 bpm, the inspiration time and the expiration timeare 2.5 seconds, respectively.

By comparing the pulse rates in cases of the respiratory periods 3 bpm,6 bpm, and 12 bpm respectively, it has been found that the period of thepulse rate variation coincides with the respiratory period. That is, arespiratory period can be calculated by measuring the period of thepulse rate variation.

As described above, change in the pulse rate correlates with arespiratory state. That is, the respiratory state estimating device 100can detect a respiratory period and a respiratory state in real time byanalyzing the change in the pulse rate.

FIG. 11 shows one example of a respiratory state estimating method. Thevertical axis indicates a pulse rate and the horizontal axis indicatestime. The respiratory state estimating unit 30 in the present exampleestimates a respiratory state from the measurement results for pulserates at continuous three points. The respiratory state estimating unit30 realizes smooth estimation of a respiratory state from thehigh-resolution continuous pulse rate.

Clock time T_(C), T_(C−1), and T_(C−2) indicates measurement times ofpulse rates respectively. The current clock time is represented by clocktime T_(C), a past clock time is represented by clock time T_(C−1), afurther past clock time is represented by T_(C−2). Also, the pulse ratesat clock time T_(C), T_(C−1), and T_(C−2) are represented by PR_(c),PR_((c−1)), and PR_((c−2)), respectively. Here, it is preferable thatthe time period from clock time T_(C−2) to clock time T_(C) be shorterthan the respiratory period of the living subject 1.

If a pulse rate is increasing continuously, the respiratory stateestimating unit 30 estimates that the living subject 1 is in theinspiration state. For example, when PR_(C)−PR_((C−1))≥0,PR_((C−1))−PR_((C−2))≥0, and PR_(C)−PR_((C−2))≥Tr, it is estimated thatthe living subject 1 is in the inspiration state. Note that Tr refers toan inspiration estimation threshold. The inspiration estimationthreshold Tr in the present example is 5 bpm. That is, when the pulserate always does not decrease between clock time T_(C−2) and clock timeT_(C), and the pulse rate increases by more than or equal to 5 bpm, itis estimated that the living subject 1 is in the inspiration state.

On the other hand, when the pulse rate decreases continuously, therespiratory state estimating unit 30 estimates that the living subject 1is in the expiration state. For example, when PR_(C)−PR_((C−1))≤0,PR_((C−1))−PR_((C−2))≤0, and PR_(C)−PR_((C−2))≤−Tr, it is estimated thatthe living subject 1 is in the expiration state. In the present example,−Tr is used as an expiration estimation threshold. However, valuesdifferent from the inspiration estimation threshold may be used as theexpiration estimation threshold. As described above, when the pulse ratealways does not increase between clock time T_(C−2) and clock timeT_(C), and the pulse rate decreases by more than or equal to 5 bpm, itis estimated that the living subject 1 is in the expiration state.

Also, when the inspiration state and the expiration state do not apply,the respiratory state estimating unit 30 estimates that the livingsubject 1 is in the instantaneous apneic state. Also, the respiratorystate estimating unit 30 may estimate that the living subject 1 is inthe instantaneous apneic state when the amount of change in the pulserate is not more than or equal to the inspiration estimation thresholdand is not less than or equal to the negative expiration estimationthreshold. Then, when the duration of the instantaneous apneic statebecomes more than or equal to the apneic state estimation threshold Ta,the respiratory state estimating unit 30 estimates that the livingsubject 1 is in the apneic state. For example, the apneic stateestimation threshold Ta may be 5 seconds.

FIG. 12 shows one example of a respiratory state estimating method. Thevertical axis indicates a pulse rate and the horizontal axis indicatestime. The respiratory state estimating unit 30 in the present exampleestimates a respiratory state from the measurement results for pulserates at continuous four points.

Clock time T_(C), T_(C−1), T_(C−2), and T_(C−3) indicates measurementtimes of pulse rates respectively. The current clock time is representedby clock time T_(C), the past clock time is respectively represented byclock time T_(C−1), T_(C−2), and T_(C−3) in order from the one closer tothe current clock time. Also, the pulse rates at clock time T_(C),T_(C−1), T_(C−2), and T_(C−3) is represented by PR_(c), PR_((c−1)),PR_((c−2)), and PR_((c−3)), respectively. Note that it is preferablethat the time period from clock time T_(C−3) to clock time T_(C) beshorter than the respiratory period of the living subject 1.

When the pulse rate varies upward continuously, the respiratory stateestimating unit 30 estimates that the living subject 1 is in theinspiration state. The upward variation refers to a variation in whichthe pulse rate tends to generally increase. For example, the respiratorystate estimating unit 30 estimates that the living subject 1 is in theinspiration state when more than or equal to two of the three pulse ratevariations between the two adjacent points (PR_(C)−PR_((C−1)),PR_((C−1))−PR_((C−2)), PR_((C−2))−PR_((C−3))) are not negative.

On the other hand, when the pulse rate varies downward continuously, therespiratory state estimating unit 30 estimates that the living subject 1is in the expiration state. The downward variation refers to a variationin which the pulse rate tends to generally decrease. For example, therespiratory state estimating unit 30 estimates that the living subject 1is in the expiration state when more than or equal to two of the threepulse rate variations between the two adjacent points(PR_(C)−PR_((C−1)), PR_((C−1))−PR_((C−2)), PR_((C−2))−PR_((C−3))) arenot positive.

Also, when the inspiration state and the expiration state do not apply,the respiratory state estimating unit 30 estimated that the livingsubject 1 is in the instantaneous apneic state. Also, the respiratorystate estimating unit 30 may estimate that the living subject 1 is inthe instantaneous apneic state when the amount of change in the pulserate is not more than or equal to the inspiration estimation thresholdand is not less than or equal to the negative expiration estimationthreshold. Then, when the duration of the instantaneous apneic statebecomes more than or equal to the apneic state estimation threshold Ta,the respiratory state estimating unit 30 estimates that the livingsubject 1 is in the apneic state. For example, the apneic stateestimation threshold Ta may be 5 seconds.

FIG. 13 shows relationship between a pulse rate variation and a pulserate variation speed. In the portion (a) of FIG. 13, the vertical axisin indicates a pulse rate (bpm) and the horizontal axis indicates time(second). In the portion (b) of FIG. 13, the vertical axis indicates thepulse rate variation speed (bpm) and the horizontal axis indicates time(second).

The subject in the present example changes his/her respiratory periodinto three stages that are 3 bpm, 6 bpm, and 12 bpm, in addition to thenormal state and the breath hold state, as with the case shown in FIG.10. As shown in FIG. 13, the period of the pulse rate variation and theperiod of the pulse rate variation speed coincide. That is, similar tothe pulse rate, the pulse rate variation speed has a correlation with arespiratory state. Therefore, the respiratory state estimating device100 can estimate the respiratory state of the living subject 1 bycalculating at least one of the pulse rate and the pulse rate variationspeed.

(Implementation 2)

The respiratory state estimating device 100 according to Implementation1 estimates a respiratory state from the pulse rate variation obtainedby the FFT. On the other hand, the respiratory state estimating device100 according to Implementation 2 estimates a respiratory state by usinga pulse wave lag time variation instead of by using the pulse ratevariation. The pulse wave lag time refers to the autocorrelation lagtime of the processing window pulse wave.

FIG. 14 shows one example of a configuration of the respiratory stateestimating device 100. The respiratory state estimating device 100according to Implementation 2 differs from the respiratory stateestimating device 100 according to Implementation 1 in that it includesthe pulse wave lag time calculating unit 40 instead of the pulse ratecalculating unit 20.

The pulse wave lag time calculating unit 40 calculates the pulse wavelag time of the living subject 1 based on the pulse wave trace signalacquired by the pulse wave signal acquiring unit 10. The pulse wave lagtime calculating unit 40 outputs the calculated pulse wave lag time tothe respiratory state estimating unit 30.

The respiratory state estimating unit 30 calculates high-resolutioncontinuous pulse wave lag time variation from the input pulse wave lagtime. The continuous pulse wave lag time variation is traced by plottingperiodically measured pulse wave lag times. The respiratory stateestimating unit 30 estimates the respiratory state of the living subject1 based on the continuous pulse wave lag time variation.

FIG. 15 shows one example of a configuration of a pulse wave lag timecalculating unit 40. The pulse wave lag time calculating unit 40includes a bias variation eliminating unit 41, an autocorrelationcalculating unit 42, and a lag time calculating unit 43.

The bias variation eliminating unit 41 eliminates a bias variationcomponent from the input processing window pulse wave. The means foreliminating the bias variation component may be a first-orderdifferentiation of the processing window pulse wave. Also, the biasvariation eliminating unit 41 may include the high-pass filter. Notethat the pulse wave lag time calculating unit 40 does not have toinclude the bias variation eliminating unit 41 when the processingwindow pulse wave is input in a state where bias is stable. The biasvariation eliminating unit 41 outputs the processing window pulse waveobtained by eliminating the bias variation component to theautocorrelation calculating unit 42 as a window processing performedpulse wave.

The autocorrelation calculating unit 42 calculates an autocorrelationcoefficient of the input window processing performed pulse wave. Theautocorrelation calculating unit 42 outputs the calculatedautocorrelation coefficient to the lag time calculating unit.

The lag time calculating unit 43 outputs a pulse wave lag time based onthe autocorrelation coefficient of the window processing performed pulsewave output by the autocorrelation calculating unit 42. The pulse wavelag time is time from some autocorrelation coefficient peak to the nextautocorrelation coefficient peak.

FIG. 16 shows one example of an algorithm of a signal processing in thepulse wave lag time calculating unit 40. The processing window pulsewave input to the pulse wave lag time calculating unit 40 is ahigh-reliability pulse wave obtained by eliminating unnecessarycomponents by means of the pulse wave signal acquiring unit 10. Thepulse wave lag time calculating unit 40 performs step S301 to step S303to calculate an accurate pulse wave lag time by using the processingwindow pulse wave that has been segmented out.

FIG. 17 shows one example of a calculation method of a pulse wave lagtime. The portion (a) of FIG. 17 shows pulse wave trace intensity. Theportion (b) of FIG. 17 shows a pulse wave trace speed. The portion (c)of FIG. 17 shows change in autocorrelation coefficient over time. All ofthe horizontal axes in the portion (a) of FIG. 17 to the portion (c) ofFIG. 17 indicate the number of samples in the case where the samplingrate is 30 Hz.

At step S301, the bias variation eliminating unit 41 performsfirst-order differentiation processing on the processing window pulsewave. Thereby, the window processing performed pulse wave with the biasvariation eliminated is obtained. For example, the graph of the pulsewave trace intensity as shown in the portion (a) of FIG. 17 turns to thegraph of the pulse wave trace speed as shown in the portion (b) of FIG.17 through the first-order differentiation. Note that the bias variationeliminating unit 41 may eliminate the bias variation by subtracting thegenerated approximate component of the bias variation from theprocessing window pulse wave. A curve obtained by approximating a biasvariation component having a frequency lower than the pulse period maybe used for subtracting the approximate component of the bias variation.

At step S302, the autocorrelation calculating unit 42 calculates theautocorrelation coefficient of the window processing performed pulsewave. For example, the portion (c) of FIG. 17 is an example ofcalculating the autocorrelation coefficient of the pulse wave tracespeed shown in the portion (b) of FIG. 17. An autocorrelation functionwhich is used for a general signal processing technique may be used forcalculating the autocorrelation coefficient. The autocorrelationcalculating unit 42 can calculates the autocorrelation coefficient peakadjacent to some autocorrelation coefficient peak by calculating theautocorrelation coefficient.

At step S303, the lag time calculating unit 43 calculates a pulse wavelag time by calculating the time difference between some autocorrelationpeak and its adjacent autocorrelation peak. The resolution of thecalculated pulse wave lag time depends on the sample frequency of thewindow processing performed pulse wave. Therefore, the resolution of thepulse wave lag time may be increased by performing upsampling on theprocessing window pulse wave or the window processing performed pulsewave. For example, spline interpolation is used for the upsampling.

FIG. 18 shows one example of continuous pulse wave lag time variation.The vertical axis indicates a pulse wave lag time ( 1/30 second) and thehorizontal axis indicates time (second). The subject in the presentexample changes his/her respiratory period into three stages that are 3bpm, 6 bpm, and 12 bpm, in addition to the normal state and the breathhold state.

The continuous pulse wave lag time variation is linked with theinspiration and the expiration of the living subject 1. However, thecontinuous pulse wave lag time variation is vertically inverted comparedto the pulse rate variation shown in FIG. 10 in terms of therelationship with the respiratory period. Specifically, when the livingsubject 1 inspires, the pulse wave lag time decreases, and when theliving subject 1 expires, the pulse wave lag time increases. Thus, therespiratory state of the living subject 1 can be estimated in real timeby observing increase and decrease in the pulse wave lag time.

FIG. 19 shows one example of a respiratory state estimating method. Thevertical axis indicates a pulse wave lag time and the horizontal axisindicates time. The respiratory state estimating unit 30 in the presentexample estimates a respiratory state from the measurement results forpulse wave lag times at continuous three points. The respiratory stateestimating unit 30 realizes smooth estimation of the respiratory statefrom the high-resolution continuous pulse wave lag time.

Clock time T_(C), T_(C−1), and T_(C−2) indicates measurement times ofpulse wave lag times respectively. The current clock time is representedby clock time T_(C), a past clock time is represented by clock timeT_(C−1), a further past clock time is represented by T_(C−2). Also, thepulse wave lag times at clock time T_(C), T_(C−1), and T_(C−2) arerepresented by PL_(c), PL_((c−1)), and PL_((c−2)), respectively. Here,it is preferable that the time period from clock time T_(C−2) to clocktime T_(C) be shorter than the respiratory period of the living subject1.

If a pulse wave lag time is increasing continuously, the respiratorystate estimating unit 30 estimates that the living subject 1 is in theexpiration state. For example, when PL_(C)−PL_((C−1))≥0,PL_((C−1))−PL_((C−2))≥0, and PL_(C)−PL_((C−2))≥Tr, it is estimated thatthe living subject 1 is in the expiration state. Note that Tr refers toan expiration estimation threshold. That is, when the pulse wave lagtime always does not decrease between clock time T_(C−2) and clock timeT_(C), and the pulse wave lag time increases by more than or equal tothe expiration estimation threshold Tr, it is estimated that the livingsubject 1 is in the expiration state.

On the other hand, if the pulse wave lag time is decreasingcontinuously, the respiratory state estimating unit 30 estimates thatthe living subject 1 is in the inspiration state. For example, whenPL_(C)−PL_((C−1))≤0, PL_((C−1))−PL_((C−2))≤0, and PL_(C)−PL_((C−2))≤−Tr,it is estimated that the living subject 1 is in the inspiration state.In the present example, −Tr is used as the inspiration estimationthreshold. However, values different from the expiration estimationthreshold may be used as the inspiration estimation threshold. Asdescribed above, when the pulse wave lag time always does not increasebetween clock time T_(C−2) and clock time T_(C), and the pulse wave lagtime decreases by more than or equal to the expiration estimationthreshold Tr, it is estimated that the living subject 1 is in theinspiration state.

Also, when the inspiration state and the expiration state do not apply,the respiratory state estimating unit 30 estimates that the livingsubject 1 is in the instantaneous apneic state. Also, the respiratorystate estimating unit 30 may estimate that the living subject 1 is inthe instantaneous apneic state when the amount of change in the pulsewave lag time is not less than or equal to the inspiration estimationthreshold and is not more than or equal to the positive expirationestimation threshold. Then, when the duration of the instantaneousapneic state becomes more than or equal to the apneic state estimationthreshold Ta, the respiratory state estimating unit 30 estimates thatthe living subject 1 is in the apneic state. For example, the apneicstate estimation threshold Ta may be 5 seconds.

As described above, the respiratory state estimating device 100 canestimate the respiratory state of the living subject 1 in real time byusing the pulse wave lag time. Alternatively, the respiratory stateestimating device 100 may improve the estimation accuracy of arespiratory state by combination use of the respiratory state estimationbased on the pulse rate and the respiratory state estimation based onthe pulse wave lag time.

Embodiment 1

FIG. 20 shows the respiratory state estimating device 100 according toEmbodiment 1. The respiratory state estimating device 100 in the presentexample is equipped on a smartphone 4. The smartphone 4 includes acamera 5 and a display 6. The smartphone 4 is one example of a portabledevice, and the respiratory state estimating device 100 may be equippedon a mobile phone, a touchpad, or the like.

The camera 5 optically acquires a video of the subject 2. The camera 5is one example of the video acquiring unit 11. The camera 5 in thepresent example acquires the video including a single portion of thesubject 2. The single portion of the subject 2 in the present example isa nose 3. Also, the camera 5 may detect the movement of the singleportion in the subject 2, and perform shooting, following the portion.For example, when the single portion of the subject 2 is moving towardthe outside of the imaging region of the camera, the camera 5 followsthe single portion by controlling pan, tilt, zoom or the like of thecamera 5 so that the single portion can be included in the imagingregion of the camera 5.

The display 6 displays the respiratory state of the subject 2 estimatedby the respiratory state estimating device 100. The display 6 may beprovided outside the smartphone 4. The subject 2 can get to know therespiratory state displayed on the display 6 in real time as if he/sheis exposed to live information.

Although the respiratory state estimating device 100 in the presentexample uses the video of the nose 3 of the subject 2, it may use thevideo of the fingertip of the subject 2. For example, the respiratorystate estimating device 100 acquires the video of the fingertip by usingoptical fingerprint sensor provided on the back side of the smartphone4. Also, the single portion of the subject 2 is not limited to the nose3 and the fingertip. The hemoglobin concentrations at the nose 3 and thefingertip are high because capillaries are concentrated there. For thisreason, by using the video of the nose 3 and the video of the fingertip,the extraction sensitivity of the pulse wave information and thecalculation accuracy of the pulsebeat information are enhanced.Furthermore, the pulse wave information may be extracted by using asingle photoplethysmogram meter put on the fingertip.

In this way, because the respiratory state estimating device 100 in thepresent example extracts the pulse wave information optically andoutputs the respiratory state, less burden is imposed on the subject 2.Further, because the respiratory state estimating device 100 in thepresent example is configured to extract the pulse wave information froma video, the respiratory state can be estimated without contacting andconstraining the subject 2. Note that if there are multiple peoplewithin the video of the subject 2, the respiratory state estimatingdevice 100 can estimate the respiratory states of the multiple people atthe same time.

Embodiment 2

FIG. 21 shows the respiratory state estimating device 100 according toEmbodiment 2. The respiratory state estimating device 100 in the presentexample is mounted within a wristband type PPG sensor 7. Also, a part ofthe respiratory state estimating device 100 may be mounted within thesmartphone 4 which can communicate with the wristband type PPG sensor 7.

The wristband type PPG sensor 7 is one example of a wearable deviceutilizing a photoplethysmogram sensor. The wristband type PPG sensor 7includes a light emitting diodes 8 and a photo diode 9. The lightemitting diodes 8 irradiates the wrist portion of the subject 2 withlight. The photo diode 9 detects the light after absorption byhemoglobin in the subject 2. Thereby, the wristband type PPG sensor 7optically acquires the pulse wave signal including information about thechange in the blood flow volume of the subject 2.

The wristband type PPG sensor 7 sends the video of the subject 2 or theestimated respiratory information to the smartphone 4 wirelessly.Wireless networks such as BlueTooth (registered trademark) and Wi-Fi(registered trademark) are used for wireless communication. When therespiratory information is input from the wristband type PPG sensor 7,the smartphone 4 displays the respiratory information on the display 6.Also, when the video of the subject 2 is sent from the wristband typePPG sensor 7, the smartphone 4 may estimate the respiratory informationbased on the video. Note that when the wristband type PPG sensor 7includes the display 6 and the respiratory state estimating device 100,the respiratory information may be displayed on the display 6 which thewristband type PPG sensor 7 includes.

FIG. 22 shows one example of fixed resampling using a lightingapparatus. In FIG. 22, a mark O indicates a fixed sampling rate and amark X indicates a video sampling rate.

The fixed sampling rate refers to an ideal frequency at which therespiratory state estimating device 100 acquires a video. For example,the respiratory state estimating device 100 acquires a video at a fixedsampling rate of 30 Hz.

The video sampling rate refers to an actual sampling rate which therespiratory state estimating device 100 acquires. For example, when therespiratory state estimating device 100 is equipped on a mobile terminalsuch as the smartphone 4, fluctuation occurs in the video sampling rate.For this reason, discrepancy is caused between the video sampling rateand the fixed sampling rate. Also, when the fluctuation occurs in thevideo sampling rate, the accurate time at which a pulse rate is acquiredcannot be learnt, and the estimation accuracy of the respiratoryinformation is reduced.

On the other hand, although the light emitted by a lighting apparatusdriven by an AC power source is not perceived by human eyes, itaccurately operates at a fixed luminance frequency. Also, the videoacquired by the respiratory state estimating device 100 includesinformation required for calculating the phase of the lightingapparatus. The phase of the lighting apparatus can be calculated fromthe intensity of the reflected light of the lighting apparatus in apredetermined region. The predetermined region may be a partial regionof an object included in the video. The predetermined region preferablydoes not move. Also, the respiratory state estimating device 100 mayshoot the light of the lighting apparatus directly instead of thereflected light of the lighting apparatus. For example, the respiratorystate estimating device 100 calculates the maximum intensity and theminimum intensity of the reflected light in the predetermined region inadvance. Thereby, the respiratory state estimating device 100 cancalculate the phase of the lighting apparatus from the video bymeasuring the intensity of the reflected light in the predeterminedregion. That is, if the video sampling rate is different from the targetphase, the phase of the video can be corrected based on the phase of thelighting apparatus. As seen above, the respiratory state estimatingdevice 100 can improve the estimation accuracy of the respiratoryinformation by correcting fluctuation of the video sampling rate withthe luminance frequency of the lighting apparatus. In other words, therespiratory state estimating device 100 can use the lighting apparatusthat has been photographed and included within an image as a referenceclock.

As described above, the respiratory state estimating device 100 canestimate the respiratory state of the living subject 1 optically with ahigh degree of accuracy. Because the respiratory state estimating device100 calculates a pulse rate optically from the portion of the livingsubject 1, it can estimate the respiratory state without imposing anyburden on the living subject 1. On the other hand, when a pulse rate iscalculated optically from the portion of the living subject 1, alow-frequency noise is easily included due to the movement of the livingsubject 1. However, because the respiratory state estimating device 100uses a higher frequency region than the respiratory state estimatingmethod using a baseline variation, noises are easily eliminated.

Also, because the respiratory state estimating device 100 calculates apulse rate every shift amount of the overlap time of the window signal,the pulse rate can be detected at an arbitrary time. That is, therespiratory state can be estimated even in a state where it is not knownwhether a next pulse is coming or not. The respiratory state estimatingdevice 100 can estimate the respiratory state in real time withoutdepending on a pulse rate. That is, the respiratory state estimatingdevice 100 can realize the smooth detection of respiration. A real timemeasurement of respiratory state can be applied to applications thatinduce the respiration method for the living subject 1 in real time.

FIG. 23 shows one example of a hardware configuration of a computer 1900according to the present embodiment. The computer 1900 according to thepresent embodiment includes a CPU peripheral unit having a CPU 2000, aRAM 2020, a graphic controller 2075 and a display device 2080interconnected by means of a host controller 2082, an input/output unithaving a communication interface 2030, a hard disk drive 2040 and aCD-ROM drive 2060 connected to the host controller 2082 by means of aninput/output controller 2084, and a legacy input/output unit having aROM 2010, a flexible disk drive 2050 and an input/output chip 2070connected to the input/output controller 2084.

The host controller 2082 connects the RAM 2020 to the CPU 2000 whichaccesses to the RAM 2020 with a high transfer rate and the graphiccontroller 2075. The CPU 2000 operates based on programs stored in theROM 2010 and the RAM 2020, and controls each unit. The graphiccontroller 2075 acquires image data the CPU 2000 or the like generateson a frame buffer provided within the RAM 2020, and displays the imagedata on the display device 2080. Alternatively, the graphic controller2075 may include therein the frame buffer that stores the image datagenerated by the CPU 2000 or the like.

The input/output controller 2084 connects the host controller 2082 tothe communication interface 2030, hard disk drive 2040 and CD-ROM drive2060 that are relatively high-speed input/output devices. Thecommunication interface 2030 communicates with other devices via anetwork. The hard disk drive 2040 stores programs and data the CPU 2000within the computer 1900 uses. The CD-ROM drive 2060 reads programs ordata from the CD-ROM 2095 and provides them to the hard disk drive 2040via the RAM 2020.

Also, the ROM 2010, the flexible disk drive 2050 and the input/outputchip 2070 that are relatively low-speed input/output devices areconnected to the input/output controller 2084. The ROM 2010 stores aboot program the computer 1900 executes at the time of start-up, aprogram which depends on hardware of the computer 1900, and/or the like.The flexible disk drive 2050 reads programs or data from the flexibledisk 2090 and provides them to the hard disk drive 2040 via the RAM2020. The input/output chip 2070 connects the flexible disk drive 2050to the input/output controller 2084, and also connects various types ofinput/output devices to the input/output controller 2084 via, forexample, a parallel port, a serial port, a keyboard port, a mouse portor the like.

The programs provided to the hard disk drive 2040 via the RAM 2020 arestored in a recording medium such as the flexible disk 2090, the CD-ROM2095 or an IC card and are provided by the user. The programs are readout from the recording medium, installed in the hard disk drive 2040within the computer 1900 via the RAM 2020 and executed on the CPU 2000.

The programs which are installed in the computer 1900 and cause thecomputer 1900 to function as the respiratory state estimating deviceinclude a pulse wave signal acquiring module, a pulse rate calculatingmodule, and a respiratory state estimating module. These programs ormodules encourage the CPU 2000 or the like to cause the computer 1900 tofunction as the respiratory state estimating device respectively.

Information processing described in these programs functions as thepulse wave signal acquiring unit 10, the pulse rate calculating unit 20,and the respiratory state estimating unit 30 that are specific means inwhich software and the various types of hardware resources describedabove cooperate as a result of reading the programs into the computer1900. Then, with these specific means, the unique respiratory stateestimating device 100 appropriate for an intended use is structured byrealizing operation or processing of information appropriate for theintended use of the computer 1900 in the present embodiment.

Also, the programs which are installed in the computer 1900 and causethe computer 1900 to function as the pulse wave measurement deviceinclude a pulse wave signal acquiring module, a pulse rate calculatingmodule, and a respiratory state estimating module. These programs ormodules encourage the CPU 2000 or the like to cause the computer 1900 tofunction as the pulse wave measurement device respectively.

Furthermore, information processing described in these programsfunctions as the pulse wave signal acquiring unit 10, the pulse wave lagtime calculating unit 40, and the respiratory state estimating unit 30that are specific means in which software and the various types ofhardware resources described above cooperate as a result of reading theprograms into the computer 1900. Then, with these specific means, theunique respiratory state estimating device 100 appropriate for anintended use is structured by realizing operation or processing ofinformation appropriate for the intended use of the computer 1900 in thepresent embodiment.

Also, the programs which are installed in the computer 1900 and causethe computer 1900 to function as the pulse wave measurement deviceinclude a pulse wave signal acquiring module, a pulse wave lag timecalculation module, and a respiratory state estimating module. Theseprograms or modules encourage the CPU 2000 or the like to cause thecomputer 1900 to function as the pulse wave measurement devicerespectively.

As an example, when communication is performed between the computer 1900and an external device or the like, the CPU 2000 executes acommunication program loaded on the RAM 2020, and instructs thecommunication interface 2030 to perform a communication processing basedon the processing content described in the communication program. Underthe control of the CPU 2000, the communication interface 2030 reads outsend data stored in a send buffer region or the like provided on astorage device such as the RAM 2020, the hard disk drive 2040, theflexible disk 2090 or the CD-ROM 2095 and sends it to a network, orwrites receive data received from the network into a receive bufferregion or the like provided on the storage device. In this way, thecommunication interface 2030 may transfer send/receive data between thecommunication interface 2030 and the storage device through the DMA(direct memory access) scheme, and alternatively the CPU 2000 maytransfer the send/receive data by reading out data from a storage deviceor the communication interface 2030 at a transfer source and writing thedata into the communication interface 2030 or a storage device at atransfer destination.

Also, the CPU 2000 causes all or a necessary portions to be read intothe RAM 2020 from among a file, a database or the like stored in anexternal storage device such as the hard disk drive 2040, the CD-ROMdrive 2060 (CD-ROM 2095), the flexible disk drive 2050 (flexible disk2090) through the DMA transfer or the like, and executes various typesof processing on the data on the RAM 2020. Then, the CPU 2000 writes thedata on which processing has been performed back to the external storagedevice through the DMA transfer or the like. Because in such aprocessing, the RAM 2020 can be considered that it holds the content ofthe external storage device temporarily, the RAM 2020 and the externalstorage device or the like are collectively referred to as a memory, astorage unit, a storage device or the like in the present embodiment.Various types of information such as various types of programs, data,tables, or databases in the present embodiment are stored on such astorage device to be subject to information processing. Note that theCPU 2000 can hold a portion of the RAM 2020 in a cache memory, and readfrom and write to the cache memory. Because the cache memory has a partof functions of the RAM 2020 also in such a configuration, in thepresent embodiment, the cache memory is also included in the RAM 2020,the memory and/or the storage device unless they are shown to bedistinguished.

Also, the CPU 2000 performs on the data read out from the RAM 2020,various types of processing including various types of operation,information processing, condition determination, informationsearch/replacement or the like described in the present embodiment thatare designated by an instruction sequence of the program, and writes theresult back to the RAM 2020. For example, the CPU 2000 determineswhether various types of variables shown in the present embodiment meeta condition such as being larger than, smaller than, larger than orequal to, smaller than or equal to, or equal to other variables orconstants when performing the condition determination, and if thecondition is met (or if the condition is not met), branches into adifferent instruction sequence or invokes a subroutine.

In addition, the CPU 2000 can search for information stored in a file, adatabase or the like within the storage device. For example, whenmultiple entries in each of which an attribute value of a secondattribute is associated with an attribute value of a first attribute arestored in the storage device, the CPU 2000 can search for an entry theattribute value of the first attribute of which matches a designatedcondition from among the multiple entries stored in the storage device,and can obtain the attribute value of the second attribute associatedwith the first attribute which meets the prescribed condition by readingout the attribute value of the second attribute store in the entry.

The programs or modules described above may be stored in an externalrecording medium. As a recording medium, an optical recording mediumsuch as a DVD or CD, a magneto-optical recording medium such as a MO, atape medium, a semiconductor memory such as an IC card, or the like canbe used besides the flexible disk 2090 and the CD-ROM 2095. Also, a harddisk provided in a server system connected to a dedicated communicationnetwork or the internet, or a storage device such as a RAM may be usedas the recording medium to provide the computer 1900 with programs viathe network.

While the embodiment s of the present invention have been described, thetechnical scope of the invention is not limited to the above describedembodiment s. It is apparent to persons skilled in the art that variousalterations and improvements can be added to the above-describedembodiment s. It is also apparent from the scope of the claims that theEmbodiment s added with such alterations or improvements can be includedin the technical scope of the invention.

The operations, procedures, steps, and stages of each process performedby an apparatus, system, program, and method shown in the claims,embodiment s, or diagrams can be performed in any order as long as theorder is not indicated by “prior to,” “before,” or the like and as longas the output from a previous process is not used in a later process.Even if the process flow is described using phrases such as “first” or“next” in the claims, embodiment s, or diagrams, it does not necessarilymean that the process must be performed in this order.

EXPLANATION OF REFERENCES

1: living subject, 2: subject, 3: nose, 4: smartphone, 5: camera, 6:display, 7: wristband type PPG sensor, 8: light emitting diode, 9: photodiode, 10: pulse wave signal acquiring unit, 11: video acquiring unit,12: trace signal generating unit, 13: window segmenting unit, 14: signalcorrecting unit, 20: pulse rate calculating unit, 21: window functionmultiplying unit, 22: integrating and outputting unit, 23: discretefrequency transforming unit, 30: respiratory state estimating unit, 40:pulse wave lag time calculating unit, 41: bias variation eliminatingunit, 42: autocorrelation calculating unit, 43: lag time calculatingunit, 100: respiratory state estimating device

What is claimed is:
 1. A method of estimating a respiratory state, themethod comprising: acquiring a pulse wave signal based on a portion of aliving subject; calculating an autocorrelation of the acquired pulsewave signal; determining a lag time of the pulse wave signal based onthe autocorrelation; and outputting a respiratory state of the livingsubject based on a length of the lag time of the pulse wave signal,wherein the respiratory state is output as an inspiration state when anamount of change in the lag time of the pulse wave signal is less thanor equal to a predetermined negative inspiration estimation threshold.2. The method of estimating a respiratory state according to claim 1,wherein the respiratory state is output as the inspiration state furtherwhen: a difference between a first value of the lag time of the pulsewave signal at a first clock time and a second value of the lag time ofthe pulse wave signal at a second clock time prior to the first clocktime is less than or equal to zero; a difference between the secondvalue of the lag time of the pulse wave signal at the second clock timeand a third value of the lag time of the pulse wave signal at a thirdclock time prior to the second clock time is less than or equal to zero;and a difference between the first value at the first clock time and thethird value at the third clock time is less than or equal to thepredetermined negative inspiration estimation threshold.
 3. The methodof estimating a respiratory state according to claim 2, wherein a timeperiod from the third clock time to the first clock time is shorter thana respiratory period of the living subject.
 4. A method of estimating arespiratory state, the method comprising: acquiring a pulse wave signalbased on a portion of a living subject; calculating an autocorrelationof the pulse wave signal; determining a lag time of the pulse wavesignal based on the autocorrelation; and outputting a respiratory stateof the living subject based on a length of the lag time of the pulsewave signal, wherein the respiratory state is output as an expirationstate when an amount of change in the lag time of the pulse wave signalis more than or equal to a predetermined positive expiration estimationthreshold.
 5. The method of estimating a respiratory state according toclaim 4, wherein the respiratory state is output as the expiration statefurther when: a difference between a first value of the lag time of thepulse wave signal at a first clock time and a second value of the lagtime of the pulse wave signal at a second clock time prior to the firstclock time is more than or equal to zero; a difference between thesecond value of the lag time of the pulse wave signal at the secondclock time and a third value of the lag time of the pulse wave signal ata third clock time prior to the second clock time is more than or equalto zero; and a difference between the second value of the lag time ofthe pulse wave signal at the second clock time and the third value ofthe lag time of the pulse wave signal at the third clock time is morethan or equal to the predetermined positive expiration estimationthreshold.
 6. The method of estimating a respiratory state according toclaim 5, wherein a time period from the third clock time to the firstclock time is shorter than a respiratory period of the living subject.7. A method of estimating a respiratory state, the method comprising:acquiring a pulse wave signal based on a portion of a living subject;calculating an autocorrelation of the pulse wave signal; determining alag time of the pulse wave signal based on the autocorrelation; andoutputting a respiratory state of the living subject based on a lengthof the lag time of the pulse wave signal, wherein the respiratory stateis output as an instantaneous apneic state when an amount of change inthe lag time of the pulse wave signal is not less than or equal to apredetermined negative inspiration estimation threshold, and is not morethan or equal to a predetermined positive expiration estimationthreshold.
 8. The method of estimating a respiratory state according toclaim 7, wherein the respiratory state is output as an apneic state whenthe instantaneous apneic state continues for a predetermined time.